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Abstract 

We consider the problem of pointwise estimation of multi-dimensional signals s, 
from noisy observations {yr) on the regular grid 7J^. Our focus is on the adaptive 
estimation in the case when the signal can be well recovered using a (hypothetical) 
linear filter, which can depend on the unknown signal itself. 

The basic setting of the problem we address here can be summarized as follows: 
suppose that the signal s is "well-filtered" , i.e. there exists an adapted time-invariant 
linear filter with the coefficients which vanish outside the "cube" {0, T}'^ which 
recovers sq from observations with small mean-squared error. We suppose that we 
do not know the filter q* , although, we do know that such a filter exists. We give 
partial answers to the following questions: 

- is it possible to construct an adaptive estimator of the value sq, which relies 
upon observations and recovers sq with basically the same estimation error as the 
unknown filter q^l 

— how rich is the family of well-filtered (in the above sense) signals? 

We show that the answer to the first question is affirmative and provide a numer- 
ically efficient construction of a nonlinear adaptive filter. Further, we establish a 
simple calculus of "well-filtered" signals, and show that their family is quite large: 
it contains, for instance, sampled smooth signals, sampled modulated smooth signals 
and sampled harmonic functions. 
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1 Introduction 



In this paper, we focus on tlie problem of denoising of multi-dimensional sig- 
nals. Let F = {Q, S, P) be a probability space. We consider the problem of 
recovering unknown random field (s^ = Sr{C,)) rex.'i over Z*^ from noisy obser- 

vat ions 

yr = Sr + Cr- (1) 

It is convenient for us to assume that both the signal {sr) and the noises 
are complex-valued. Besides this, we assume that the field (e,-) of observation 
noises is independent of (st) and is of the form Cr = crer, where (e,-) are 
independent of each other standard Gaussian complex-valued variables; the 
adjective "standard" means that 3?(eT-), Q'(er) are independent of each other 
N(0, 1) random variables. Our focus here is at estimating the value St of signal 
at a given location t G Z'^. 

The above problem is "classical" in statistical estimation and signal process- 
ing, and as such, has received much attention. In particular, linear estimators 
(referred as linear filters in the signal processing community) are widely used 
in the statistical literature. To be more precise, suppose that our aim is to 
recover the value sq of the signal at zero given observations {yr) on the box 
Or = {r G Z'^ : \Tj\ < T,l < j < d}. We call the estimation s of Sq linear if 
it is of the form 

t&Ot 

for some q G C(Ot), where C{Ot) is the set of complex- valued fields q = 
{qr, T G Ot} over Ot- 

The simplicity of linear estimators is responsible for their popularity in statis- 
tical signal processing. Another outstanding feature of such estimators is their 
minimax property. Suppose that the a priori information resumes to the fact 
that [sr) belongs to some convex compact set which is symmetric with respect 
to zero, let us call it S. One of the most renown results of estimation theory 
(see, for instance, [T3HIID]) states that the linear minimax estimator is, in a 
certain sense, an optimal estimator of Sq in our problem. Indeed, consider the 
following linear minimax estimation strategy: let gl"^^ be the optimal solutionPI 
to the problem 



For evident reasons such a solution exists in the situation we are interested in. 
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(here Eg stands for the expectation with respect to the distribution of (y^) 
which corresponds to the underlying signal s) . The linear minimax estimator 
s} of So is defined by 

Then 

ma,xEs{so — s^)^ < C inf raaxEs{so — sY, 

where the infimum in the right-hand side is taken over all possible estima- 
tors of So from observations (?/,-) and C is a moderate absolute constant (e.g., 
C < 1.25). In other words, the linear estimator se is a (almost) minimax esti- 
mator of sq. We would like to stress the exceptional power of the above result 
- we only need S to be convex and compact for the linear estimator to be 
minimax optimal. The evident downside of using linear minimax estimators is 
that the a priori information about the set S of signals should be as precise 
as possible to achieve descent estimation accuracy. There was a significant re- 
search on adaptive estimation in the above setting (cf [6 ,,7]). Those techniques 
allow to choose the "best" in a certains sense set which contains the signal 
from special finite families of convex sets. Another "classical" approach to 
adaptation for linear estimators has been developed in [20ll21|22|23|28j . In the 
latter approach the "form" of the filter g^^-* is considered as given in advance 
(no information about sets of signals is used in this case), and the parameter T 
(the "window width") is selected adaptively to achieve the best bias/ variance 
tradeoff. Recently, more general adaptation techniques has been studied in 
[2^fT^ . which allow to choose the best estimator from special finite families 
of available linear estimators. 

The problem we are interested in here, when posed informally, is as follows: if 
we consider the form of the filter as a "free parameter" , is it possible to provide 
an estimation procedure which is adaptive with respect to this parameter? 
In other words, suppose that a "good" filter , with a small estimation 
error exists. Then, is it possible to construct a data-driven estimation method 
which has (almost) the same accuracy as the "oracle" - a hypothetic optimal 

(T) 

estimation method which uses the "good" filter . It is natural, as it is 
common in adaptive nonparametric estimation, to measure the quality of an 
adaptive estimation routine with the factor by which the risk of the adaptive 
procedure is greater than that of the "oracle" estimator. What we look for is 
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the estimation method for which this factor is not too large. Let us consider, 
for instance, the following question: 

(?) Suppose that know that the (deterministic or random) signal (s,-)rez<* = 
(s^(^))^gzd underlying observations can be recovered from these observa- 

tions "at a parametric rate" by "linear time-invariant filtering": for a given 
T, there exists (unknown in advance) filter which recovers sq via 0{T ) 
observations around zero such that 

eLo-T. &A<0{a'T~'^). (2) 

I tSOt J 



Can we mimic this filter? 

We show that the answer to the question (?) is positive. Namely, whenever 
a discrete time signal (that is, a signal defined on a regular discrete grid) is 
well- filtered, i.e., can be recovered from its noisy observations at a parametric 
rate by a linear time-invariant filter, we can recover this signal at a "nearly 
parametric" rate without a priori knowledge of the associated filter. 

Several points should be stressed in the above claim. First, we are able to mimic 
only ideal filters ql of small /2-norm. Indeed, the relation ([2]) implies that 

the stochastic term of the error E i J2 qiVcr ] is bounded with 0(cr^T~'^), 

\tGOt ' / 

which is conceivable only if I2 = 0{T^'^/'^). This constraint is crucial, as the 
price for adaptation becomes prohibitive when the /2-norm of the ideal filter is 
much larger than 0(T~'^/^). Though this assumption seems quite restrictive, 
the family of well-filtered signals is quite wide. As we shall see later, this family 
contains also "highly oscillating" sampled modulated smooth signals, sampled 
harmonic functions, etc. 

In this paper we also treat the problem of adaptive prediction, when we are 
interested in recovering of a discrete time signal at a point t G Z'^ via noisy 
observations taken at the points {r G Z'' : — T < Tj < tj — k,} "preceding" 
the point t, with a given in advance "forecast horizon" k > 0. 

The rest of our paper is organized as follows. In Section [2] we give a formal 
definition of a well-filtered (well-predicted) signal on a d-dimensional regular 
grid (the latter, w.l.o.g., is normalized to be Z'^), and then show in Section [3] 
demonstrate that such a signal can be recovered at a nearly parametric rate 
without a priori knowledge of the corresponding "good filter" (Theorems H] 
and [5]). The underlying estimation routines (i.e., "Algorithm A" of Section [3TT] 
and "Algorithm B" of Section 13. 2p constitute a substantial extension of the 
procedures proposed in [25] and ^SB] . In Section 14.11 we demonstrate that the 
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family of well-filtered signals is pretty wide - it contains a wide spectrum of 
"basic functions" (for example, exponential polynomials) and is closed with 
respect to a number of basic operations, including modulation, taking linear 
combinations and tensor products. 

To make the exposition more readable, all proofs are collected in the appendix. 

The denoising procedures, described in this paper constitute the basic bricks 
of the construction of adaptive estimators of locally well-filtered signals, which 
we describe in the companion paper [18]. The results of [18] extend to the wide 
classes of modulated signals the results of [27|12|16p3] on spatial adaptive 
estimates of signals with inhomogeneous smoothness. 



2 Problem statement 



In order to proceed we need some notations. 

Fields over Z'^. Let C{Z'^) be the linear space of complex-valued fields r = 
{rr-.re Z'^} over Z'^. 

• Given nonnegative integer T and p G [1, oo], we define semi-norms | ■ \t,p 

/ \ i/p 

on C{Z'^) by \r\Tp = Vt\^] ■, kl = niax{|ri|, |r(i|}, with the stan- 

\\r\<T J 

dard interpretation of the right hand side when p = oo, and we set \r\p = 
limr^oo \^\t,p € M U {+cxd}. A field r G C{Z'^) with finitely many nonzero 
entries r-r is called a filter, and the smallest T such that r,- = whenever 
|r| > T, is called the order ord(r) of a filter r; we write Ct{Z'^) = {r G 
C(Z°') I ord(r) < T}. We identify a filter r with the multivariate Laurent sum 
r{zi, ...,Zd) = T.rrZl\..zY. 

T 

• We call a filter r polynomial, if the corresponding Laurent sum is a polyno- 
mial (i.e., if the entries r^- vanish when any of Tj < 0, j = 1, (i). The set of 
all polynomials is denoted P{Z'^). For integers k,T,0<k<T,we denote by 
P^{Z'^) the subspace of P{Z'^) formed by polynomials r for which the entries 
r^- vanish outside the set k < tj < T, j = 1, ...,d. 

• We denote by Aj, j = 1, ...,d, the "basic shift operators" on C{Z'^): 

(^i^)Tlv,Td ~ ^Tl,...,Tj-l,Tj-l,Tj + l,...,Td- 

Further, we use the notation for the inverse of A^-: 

(Aj r),-^^,,,^^-^ — ''"ri,...,Tj_i,rj + l,Tj + i,...,T£j- 

• Finally, we define the output of a filter r, the input to the filter being a 
field X G C(Z^), as the field r(A) X = r(Ai, A2, Aj;)^;, so that (r(A)x)t — 

J2 T^T^t-T- 



5 



Fourier transform. Let T be a nonnegative integer, let Tt be the set of roots of 
1 of the degree 2T+ 1, and let C(T^) be the space of complex- valued functions 
on = {TtY. 

• We define the Fourier transform Ft : C{'L'^) CiV^) as {Fxr){fi) = 
(2T+i)d/2 I E^'^r/^i'---Ai? = (2T+\)'^/2 ^(/^)' ^ ^ CtC^"^), where /i G T^. Note that 

= (2T+\)''/2 51] iFTr){fJ,)fJ,i^^ ...fid^'^, V(r : |r| < T). The Fourier transform 



- T 

7d\ 



allows to equip C(Z ) with semi- norms coming from the standard p- norms on 

c(r^): 

\r\T,P = IFttI, ^ \ E l(^Tr)(/i)r 
with the standard interpretation of the right hand side for p = oo. 



Now it is time to give a precise meaning to the basic question (?) of Intro- 
duction. In order to do this, we should specify our a priori knowledge of the 
constant factor hidden in O(-) and on the ranges on values of T and r where 
(ED holds true. 



2.1 Nice signals 

Since the observation noises are independent of (sr), we have 

E {\sr - {q{A)y\\'} = 2aWl + E^ {|s.(0 - (g(A)s(0).r} ; (3) 

therefore in order to ensure ([2]) , both terms in the right hand side of the latter 
inequality should be of order of T~^. This observation motivates the following 



Definition 1 Let 6 > 0, p > 1 be reals, let L be a nonnegative integer or 
+00, and let t G Z'^. Finally, let {Sr)rei.'^ = i^riO) rei^ be a random field on 

(1) [T-well-filtered signals] Let T be a nonnegative integer. We say that [sr) 
is T-well-filtered, with the parameters 9, p, L, at the point t (notation: {sr) G 
S\{9, p,T)), if there exists a filter q = q^"^^ G Ct(Z'^), \q\2 < j2T+ij^' '"^^^^/i 
reproduces (s^) in the box {r : |t — )f:| < L} with the mean square error not 
exceeding 9{2T + 1)-"^/^ : 

1/2 



max 

T:\T-t\<L 



E {\sr - {q{A)sU'}] ' <9{2T + 1)-'/'. (4) 
(2) [well-filtered signals] We say that (s^) is well-filtered, with the parameters 
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9, p, L, at the point t (we use the notation: (st) G F^(6',p)), if, for every 
integer T, < T < L, (sr) is T -well-filtered, with the parameters 6,p,L, at t. 

In the above definition we were focusing on the case of de-noising - recovering 
a well-fihered signal (s) at a point t & if via a given number observations 
"around" this point H] Another interesting problem is that of prediction^ where 
the goal is to recover st via observations Ur "preceding by a given horizon 
K e Z+" the point t, i.e., observations with Tj < tj — k, j = 1, d. 

Definition 2 Let 6 > 0, p > 1 be reals, let Tq > k, be nonnegative integers, 
L be a nonnegative integer or +oo, and let t G Z''. Finally, let {Sr)re7.'^ = 
(■Sr(O) -rezd be a random field on TL^ . 

sen 

(1) [T- well-predicted signals] Let T be a nonnegative integer. We say that (s^) 
is T-well predicted with the parameters 9, p, k, L, at the point t (notation: 
(Sr) G Q^(^,p,T)), zf there exists a filter q = g(^) G P^{Z''), \q\2 < (^f^, 
which reproduces (sr) in the box {r : |t — 1| < L} with the mean square error 
not exceeding 6{2T + l)^'^/^.- 

1/2 



max 

T:\T-t\<L 



E{\sr-{q{/^)s)r\^}]' <e{2T + ir'"\ (5) 



(2) [well-predicted signals] We say that {sr) is well-predicted, with the param- 
eters 6, p, K, Tq, L, at the point t (notation: (s^) G Pk,To,l(^> P))' /^'^ every 
integer T, Tq <T < L, (s^) is T -well-predicted, with the parameters 6, p, k, L, 
at t. 

Remark 3 Note that the quantitative description of a well-predicted field, 
when compared with the description of a well-filtered field, involves an extra 
parameter Tq - the smallest "window width" starting with which a possibility 
to predict St is postulated. In the case of well-filtered fields, this width is just 
0, in full accordance with the fact that in the de-noising problem every signal 
is 0-well-filtered, at every point t, with parameters 6 = 0, p = l, L = oo due 
to the existence of the trivial "single-point" filter q{z) = 1. 

In the sequel, we qualify as nice a signal which fulfils the requirements of 
Definition [2] or [1] above. The filters g^^-* associated, in the sense of the above 
definitions, with a nice signal {sr) as to filters certifying the "niceness" ("well- 
filterability" of "well-predictability") of the signal. 

We are about to demonstrate that in the framework, suggested by the above 
definitions, the answer to the question (?) is affirmative. I.e., a signal which is 
nice (T-well-filtered or T-well-predicted, with parameters 0, p,L = 3T) at a 
point t can be recovered at this point "at a nearly parametric rate" with no 

^ To be more precise, in the filtering literature this case is referred to as interpola- 
tion. 
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a priori knowledge of the corresponding "good Glter"; all we should know in 
advance are the parameters p and T. 



3 Main result 



We start the recovering routine for the adaptive filtering problem. 



3. 1 Adaptive filtering 



The estimator we intend to use is as follows: 

Algorithm A: Given a setup (p > 1,T) and a point t G Z'^, we build an 
estimation St[T,y] of St via observations {Ut), |t — i| < 4T, as follows: 

(1 ) When T = 0, we merely set St[0, y] = yt 

(2) When T > 0, we set St[T, y] = (0*(A)?/)t, where 0* G C2r(Z^) is an optimal 
solution to the following optimization problem: 



mm 



where y\ = {yr '■ \t — t\ < L} . 

Note that the objective in ([6]) is affected only by observations y\rp, so that our 
algorithm recovers St via (8T + 1)'^ observations "around" the point t. 

Theorem 4 Assume that the signal (sr) underlying observations is T- 
w ell- filtered, with parameters 9, p, L > 3T; (st) G S^(6',p, T) with L > ST. 
Then the mean square error of the estimate St[T, ■] of st yielded by Algorithm 
A with setup (p, T) can be bounded from above as follows: 



(j^friT 1 \nV/'< 3 g + ap^ln(2T + l) + l 
(E{\st[T,y]-st\ I) <c{d)p ^^^YpT^ ' 

c{d) = 3(2^ + 23^-1). 



In particular, if (sr) is well-filtered, with the parameters 9, p, L, at a point t, 
then for every integer T , < T < [i^/3j, the accuracy of the estimate Sj[T, 
of St yielded by Algorithm A can be bounded by ([7]). Finally, in the case of 
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deterministic (s), we have 



\st - St[T,y]\ < c{d)p' [9 + ape^] (2T + l)-'^/^, 

(8) 

e^ = a-i max \AY-'\..A''/-'''e\%T ^. 

t:\t\<2T' 

Comments: note that Theorem H] gives an affirmative answer to the question 
(7). Indeed, let a signal (sr) admit, for some T, a filter-type estimate Sr = 
{q*{A)y)r with "window width" T (i.e., with q* e Ct{'^'^)) and with the 
mean square error which, in an 0(T)-neighborhood of a point t, is of the 
"parametric" order O (a{2T + l)-^/^^: 

max E {\Sr — Sr\ \ < = — — -777- (9) 

r:\r-t\<2.T V ^ ^1 i " (2T+ l)'*/2 ^ ' 



with some known /x > 1. We do not know what is this estimate, although do 
know that it exists (i.e., know the associated T, /x), and we want to recover 
St from observations y\rp nearly as well as if we were using our hypothetic 
estimate St- Theorem H] says that Algorithm A basically achieves this goal. 
Indeed, from ([3]), ([9]) it follows that \q*\2 < (2T+i)d/2 ^"^^ i^r) G S|j.(cr/i, /i, T). 
Applying Theorem H] with p = fi, 6 = cr/i, L = 3T, we conclude that with the 
estimate yielded by Algorithm A, the mean square error of recovering St does 



not exceed 0(l)/i^ 1 + y ln(2T + 1) k. We see that as far as the dependence 
on ^'observation time" T'^ is concerned, the estimate yielded by Algorithm A is 
just by a logarithmic in T factor worse than the estimate St we wish to mimic. 
In the literature on nonparametric estimation the bounds as in Theorem H] are 
often referred to as oracle inequalities. Since the pioneering work [T] a number 
of oracle inequalities have been established for a wide variety of estimation 
problems (cf. the papers [19], [2], [3], [9], [11], [5] among many others). In 
that context one refer to the filter g, which certifies the niceness of the signal, 
as the oracle, and the bound ([7]) describes the ability of a particular adaptive 
method (Algorithm A above) to reproduce the oracle. 

Note that the "upper bound" of Theorem H] may be compared to the lower 
bound of Theorem 2 of [17] for the 1-dimensional situation. The latter re- 
sult states that one can exhibit a family of signals which 1) each member 
of the family can be recovered with the rate 0{-^) using the corresponding 
certifying filter; 2) the rate of estimation of signals from the family using the 
observation ([1]) is at best O i^^P^\f^-)- In other words, it states that the factor 



py\n{2T + 1) is an unavoidable "price" for adaptation. When comparing the 
result of Theorem H] to that lower bound, we observe an extra factor p^ > 1 
in the corresponding upper bound ([7]). By now we do not know if this extra 
factor can be completely eliminated. Nevertheless, in light of these results, we 
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can claim that recovering of signals with certifying filter of large /2-norm is a 
rather desperate task - the price for adaptation is then proportional to p ^ 1 
in this case. 



3.2 Adaptive prediction 



We now turn to the problem of adaptive prediction. The predictor we intend 
to use is as follows: 

Algorithm B: Given a setup {p > 1,k,T) and a point t G Z'', we build a 
prediction St[K, T, y] of St via observations {Vt), k <tj — tj < AT, j = 1, d, 
as St[n,T,y] = {ip^{A)y)t, where G P2t(^^) ^'^ optimal solution to the 
following optimization problem: 

{ .l^r-...A.-'(i -^(A)),|;,,^ : < pl^}; (10) 



where y^^ ^ = {Vr ■ k < tj - tj < L, j = 1, rf} . 

Note that the objective in (|TOl) is affected only by observations 4^, so that 
our algorithm recovers St via (4T — k, + lY observations "around" the point 
t. 

Theorem 5 Assume that the signal {sr) underlying observations ([1]) is T- 
well-predicted, with parameters 6, p, k, L > 3T: (sr) G Q^^(6', p, T) with 
L > 3T. Then the mean square error of the estimate St[n,T, •] of St, provided 
by Algorithm B with setup {p,K,,T), can be bounded from above as follows: 



[E{\st[^,T,y]-s,\ \} <c{d)p ^ , ^^^^ 

c{d) = 3(2^ + 23^-1). 



In particular, if (s^) is well-predicted, with the parameters 6, p, n, Tq, L, at 
a point t, then for every integer T , Tq < T < [i^/3j, the accuracy of the 
estimate T, y] of St yielded by Algorithm B can be bounded by ffTTl) . 
Finally, in the case of deterministic {s), we have 

\st - st[T. y]\ < c{d)p^ [9 + apey (2T + ly^/^ 



The proof of Theorem [5] is identical to that of Theorem HI 
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4 Families of nice signals 



When applying Algorithms A, B and Theorems HI [5|, the crucial question is how 
to recognize niceness. We are about to give a partial answer to this question. 



4-1 Calculus of nice signals 



Our current goal is to understand how wide are the families of nice signals, and 
our plan is as follows: (a) we list a number of operations which preserve the 
property in question, and (b) we present a list of examples of signals possess- 
ing the property. Applying to "raw materials" from (b) operations from (a), 
one can produce a wide variety of nice signals. Here is a sample of operations 
preserving niceness of signals. 

I. "Scale" of nice signals. We start with the following evident observation: 
p' > p,e' >e,L' < L ^ Fi{9, p) C Fi,{e', p') and p' > p.O' > 6, k' < > 

II. Taking linear combinations. Our next observation is that a linear combi- 
nation of "good" signals is again good, with properly updated parameters: 



Proposition 6 (i) Let e F\{9j,pj), and let Xj E C be random variables 
independent of (s^) and such that i?{|Ajp} < oo, j = 1, m. Then 



= (2m - l)'^/22-Vi-P™ £ (13) 

j=i '"J 

p+ = (2m- l)'^/22>i...p^,L+ = [L/2J. 



In the case of m = 1, one can set p~^ = pi, 6^ = |Ai|6'i, = L. The filters 
certifying the well-filterability of (sr) can be chosen to be independent of the 
coefficients Xj. 

(a) Let {sD G P'^^^j ^{6j, pj) , j = l,...,m, and let Xj E C be random variable 
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independent of (s^) and such that E{\Xj 



} < OO; J 



1, m. Then 




m 



9+ 



m 

(2m - lY/-'2^-^pr...pm E 



(14) 



P 



,+ 



(2m-l)'^/22"pi...p^,«;+ = 




m max Tj, 



l<j<m 



LL/2J. 



/n ^/le case of m — 1, one can set p'^ — pi, 9^ — |Ai|^i; — k, Tq — Tq, 
L'^ = L. The filters certifying the well-predictability of (sr) can be chosen to 
be independent of the coefficients Xj . 

III. Modulation and conjugation. Next we notice that the famihes of nice 
signals are closed w.r.t. "modulation" and conjugation: 

Proposition 7 (i) Let (s^) G F\{9,p), and let uj E MJ^, G M 6e determin- 
istic. Then the signal {sr = exp{i[ci;"^r + (f)]}sr)rezd belongs to F\^{9,p) along 
with (sr), and the signal {sr — s7)r (a is the complex conjugate of a G C) 
belongs to F\^{9, p). 

(a) Let {sr) G PrTolI^'P)' '^^'^ ^ ^ ^'^^ G M &e deterministic. Then 
the signal (s^ = cyi^{i[uj^T + 4>\}sr)r&z<i cs/so belongs to Pk,To,l(^' P)? 
signal {sr = s7)r belongs to P^To li.^^ p)- 

IV. Lifting. We are about to show that a nice signal in a dimension d < can 
be viewed as a nice signal, with properly updated parameters, in a dimension 
(i+ > d: 

Proposition 8 (i) Let 1 < d < d~^ , and let (s^)^g2d be a signal which is well- 
filtered, with parameters 9, p, L, at a point t G Z'^. Then the signal (s+ = 

STi,...,Td) well-filtered, with the parameters 9^ = (2L + \^^'^^~'^)l'^9 , p'^ = p, 
L^ = L at every point G Z'^^ such that {t^, ■■■,t^) = t. 
(a) Let 1 < d < d'^, and let (sT-)rez<* be a signal which is well-predictable, with 
parameters 9, p, k, Tq, L, at a point t G Z'^. Then the signal (s^^^. .^^-^^ = ^n.-.-.Td) 
is well-predictable, with the parameters 9^ — (2L + l)(^'''~'^)/2^^ p+ = [2k + 



l~^(d+-d)/2p^ «;+ = n, T^ = To, L+ = L, at every point t+ G Z'^+ such that 
{tt,...,t^)^t. 



V. "Tensor product". Let d — d' -\- d" with positive integers d' , d", so that 
Z*^ = Z'^' X Z'^". Given random fields (s'^'iO) r'ezd' , r"eEd" , we define 

their tensor product as the field (sr(0 — (0 K" (0) t=(t', T")ei.d ■ 



Proposition 9 (i) Let {s'AO)r'e.d' e Fi{0,p'), {s';„{0) r"e.d" e F*;'(0,p")- 
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Then (sr) G Ff'*"^(0, pV). 

(n) Let {s'AO).Y e P1to,l(0,p'), {s'^.^r'^' ^ KtoA^^p")- Then 

(^.)ePi*i?';i(0,py)- 



4-2 Examples of nice signals 

I. Exponential and algebraic polynomials. Let us define an exponential poly- 
nomial (sr) on Z*^ as a Gnite sum of exponential monomials cr° expjcu-^r} = 
CTi^ ...T^'' exp{uj'^T} witli nonnegative multi-indices a and u G C'': 

Sr = ^C,T'^^'^exp{uJ^{i)T}, (15) 

£=1 

wliere a;(£) and a{i) are deterministic, and q may be random. Given an ex- 
ponential polynomial (sr) on Z^, we define its partial sizes Nj, j = l,...,d, 
as follows: let rrij be the maximum of the degrees aj{i), i = I, ...,M, of the 
variable tj in the monomials of the sum (ITSl) . and Mj be the number of dis- 
tinct from each other complex numbers among the "partial frequencies" ujj{i): 
Mj = CardOj, = {ujj{i) : 1 < £ < M}. The j-th partial size Nj{s) of 
exponential polynomial f|T5l) is, by definition, the integer [mj + l)Mj. For ex- 
ample, with all frequencies equal to 0, an exponential polynomial becomes an 
algebraic polynomial, and its j-th size is by 1 larger than the degree of the 
polynomial w.r.t. j-th variable tj. 

Proposition 10 Let (sr) be an exponential polynomial on Z"' of partial sizes 
Ni, Nd- Then for all t E one has 

(sr) e FUO, Pd{N,,...,N,)), p,{N,,...,N,) = n[(2iV, - 1)1/223A^^/2], (16) 

i=i 

and the filters q^^^ certifying this inclusion can be chosen to be dependent 
solely on T and on the collection of d sets Oj = {ujj{£) : 1 < £ < M} of 
partial frequencies. 

Remark 11 ^4 major shortcoming o/ ffTB]) is a dramatic growth of pd{N, N, N) 
with N and d. In several important cases, better bounds for p can be found. 
For example, an algebraic polynomial of degree m in every variable 

a>0,\oi\<m 
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belongs to F^(0, (16m)'^) for every t, and the filters g^'^^ certifying this inclu- 
sion can be chosen to depend solely on T, d, m. 

II. Solutions to homogeneous difference equations and Iiarmonic functions. 
Consider a difference operator D: 

k 

{'Df)r = T.^fJr-aiiy, (18) 
i=l 

here a{l), ...,a{k) G Z'^ and wi,...,Wk G C. For a positive integer and 
t e Z'^, let 

5^ = {r G I |r - t| < N}, 5^(D) = {r E B% \ r + a{£) E B%, i = 1, k}, 
H*v(D) = {{s) E CiZ'') I Sr = (Ds), Vr G 5^(D)}. 

For example, with 

(19) 

E = ±l 

the linear space H^(D) is the space of fields which are "discrete harmonic" on 
that is, Sr = ^ E s^,,...,^^_,,^,+e,T-,+i,...,Td for all r with |r - t| < iV - 1. 

i— 1, d 



i=l,...,d 
e = ±l 

Let us call a difference operator D regular., if it possesses the following prop- 
erties: 

R.l The vectors {a;(£)}i<£<fc span the entire W^] 

R.2 The coefficients Wi = pe exp{i(j)i} {pi > 0, (pe E M) are nonzero, and 

(a) E P£ < 1; (6) E P£a(£) = 0. (20) 
e=i e=i 



For example, the averaging operator (fT9|) and its degrees are regular. 
It turns out that the solutions of homogeneous difference equations with reg- 
ular difference operators are well-filtered: 

Proposition 12 Let JD be a regular difference operator. Then there exists a 
constant c = c(D) > such that 

ViV>0: H*v(D)cF;,^j(0,c-1). (21) 



As a nontrivial application example for Proposition [121 consider the families 
of random fields defined as follows. Let d < A, M he a positive integer, and R 
be a positive real. Consider the family H"'"(M) of all deterministic continuous 
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functions / on which arc harmonic in the interior of the box — {x & 

M*^ : \xj\ < 2M,j <d}: (^E f(x) ^ 0, x E intD^j^. Now let H+(M,i?) 

be the family of random functions / such that all realizations of a function 
belong to H+(M) and, besides this, 2m} ^ where ||/||oo,2M is the 

uniform norm on D2m- Restricting functions / from H"'"(M, R) on Z*^, we get 
a family of random fields H(M, R) on Z*^. 

Proposition 13 Let d < A, M be a positive integer and R> be a real. For 
an appropriately chosen absolute constant c > 0, for all deterministic fields 
{sr) e H(M, R) one has 

\t\ <cM,L<cM=^ {sr) e F\{c-^R, c"^), (22) 

and the filters q^'^^ certifying the above inclusion can be chosen depending solely 
on d, T. 

4.3 Basic example of well-predicted signal: quasi-stable exponential polyno- 
mial 

Let us define a quasi-stable exponential polynomial (st) on Z"* as an exponen- 
tial polynomial 

M 

s.-Ec£r"(')exp{a;^(£)r} (23) 

where all partial frequencies ujj{i) satisfy the restriction 'Si{ujj{i)) < 0. For 
example, an algebraic polynomial (partial frequencies arc zero) and a trigono- 
metric polynomial (partial frequencies are imaginary) are quasi-stable. 

Proposition 14 Let {Sr) be a quasi-stable exponential polynomial on IJ^ of 
partial sizes Ni, ...,Nd. Then for every integer k >0 and all t e Z"^ one has 

p„,,(7Vi, TV,) ^ fi [{2N, - 1)V22^. (max[2, 2k + 1])^^/'], (24) 

Tq — K max Ni 

i<j<d ■' 

and the filters q^"^^ certifying this inclusion can be chosen to be depending solely 
on T,K and on the collection of d sets Oj — {cUj(£) : 1 < £ < M} of partial 
frequencies. 
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5 Appendix 



5. 1 Preliminaries 



Norm relations. Let us list several evident relations between the introduced 
semi-norms on C(Z'*). 
• [Parseval equality]: 



{r,s)T= E rts-t= J2 iFTr){i^){FTs){ii) = {FTr,FTs)T, (25) 



where a is the complex conjugate of a G C; in particular, 

|r|r,2 = \r\T^2', (26) 



A useful corollary of Parseval's equahty combined with the fact that |g|r,p 
\q\T,p is the relation 



\t\<T 



< \<T,l\b\T,oo- (27) 



• [Norms of convolutions of filters] 

r,s e C(Z"') ^ \r{zi, ...,Zd)s{zi, , ...,Zd)\p < \r\i\s\p; (28) 

• [Relations between | • | and | • |*]: for p, g G [1, oo] one has 

\r\*T^p < (2T + i)m/p-m+Hi/^-m+] |^|^^^^ a+ = max[a, 0]; (29) 
ord(r) +ord(s) <T \r{zi, Zd)s{zi, Zd)\T^p < |r|i|s|^_p. (30) 

Useful fact. In the sequel, we need the following simple and well-known fact: 

Lemma 15 Let fj — + ir]j, < j < N , be a sequence of N standard Gaus- 
sian complex-valued random variables, not necessarily independent of each 
other. Then 

[^{max l/,f}]V2< V21niV + 2; 

0<3<N ^g-j^'^ 

P{^mi^ \fj\ >u+ V2 InN} < exp{-'u72} V-u > 0. 
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Proof. We have 
ijj{r) = P{^max^ > r} < min[l, A^exp{— r^/2}] =^ 

P{^max^ 1/^1 >u + V2 IniV} < iVexp{-(M + V2 lniV)V2} < exp{-MV2}; 

oo oo \/2\nN 

E{ max = — / r'^dip{r) = 2 / rip{r)dr < 2 / rtir 

o<i<A^ 

oo 

+2A^ / rexp{-rV2}c;r = 21nA^ + 2. □ 

V21nAf 



5.^ Proof of Theorem^ 

W.l.o.g., we may assume that t = 0. We denote by q* the filter associated with 
{sr) via the description of the inclusion (s^) G 832.(6', p, T). Let us set 

|g*|2 = p(2T + l)-'^/2. ^ = ^(2r+l)-^/2 ^32) 
so that 

s = q*{A)s^ max E{\sr-Sr\^} <K^. (33) 

t:|t|<3T 

Finally, let 

max \^l\.A'Xe\lToo^ (34) 

T:|r|<2T' ^ \2.1,ooi \ I 

and let be the optimal solution, used in Algorithm A, of the optimization 
problem ([6]). 

1°. We start with simple technical lemma: 

Lemma 16 Let r(zi, 2^) = (q'*(-2i, -^d))^- Then r G C^t^^^ possesses 
the following properties: 

\r\2 < \r\*2T,i < 2''/'p'(2T + l)-'^/^. (35) 
|r|i<p2; (36) 



1/2 



P{|(1 - r(A)).|^,.,2}J < k(p + 1)(4T + 1)'^/^; (37) 
(l-r(A))y|*^^^ < |(l-r(A)).|2T,2 + (l + p2)eT (38) 

1 /2 

E |(|(1 - r(A))y|*^ J'}] < a(l + p2)^4rfln(4T + l) + 2 ^^^^ 

+fi:(p+l)(4T+l)^/2_ 
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(40) 



Proof. (ESI); We have 



r 



(4T+l)d/2 

(4T + l)'^/'(|g*|;r,2)' = (4T + l)'^/'|gl2T,2 < 2'^/V2(2T + 1)-^/^^ 



Since \r\2 = \r\2T,2 = \^\2t.2 — kl2T.i) pSj) follows. 

(Eg): We clearly have |r|i < \q*\l < ((2r + l)^/2|^*y2 ^ ^2^ 

^37\] : Let /i = (1 — g*(A))s, so that by virtue of (sr) G S°(6', p, T) and in view 

of the origin of q* we have 

max E\\hr\'^} < k^. (41) 

t:\t\<3T I-I I J - 



Setting g = {1 — r(A))s, we have 

= ((1 + g*(A))(l - g*(A))s). = ((1 + q*{A))h)r = K + (g*(A)/i). 

\T':\r'-T\<T j 

applying (HTl) and taking into account that |g*|2 = p(2T + l)^'^/^, we come to 
max £;{|((1 - r(A))s).p} < [k(p + 1)]^ , (42) 



and (1371) follows. 
(Eg), (Eg); We have 

1(1 - r(A))2/|*^,^ < 1(1 - r(A)).|*^,^ + |(1 - r(A))e|^^,^ 

< 1(1 - r(A)).|*^,2 + 1(1 - rm^Wr^oo = 1(1 - r(A)).|2T,2 + |(1 - r(A))e|^^,, 

< |(l-r(A)).|2T,2 + |e|^r,^+ E |r.||AI\..A^''e|;^,^ 

t:\t\<2T 

< |(l-r(A))s|2T,2 + (l + |r|i) max |A[\..A^^e|^^,^. 

t:\t\<21 
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The resulting inequality combines with (l36i) to yield (l38l) . Further, from the 
resulting inequahty and (137|) it follows that 



E\{\{l-r{A))y\y, 



,00 



1/2 



< fi:(p+ 1)(4T + + (1 + |r|i)(^E| max^|A[\..A 



Td I* 

d ^\2T,oo 



2 - . 1/2 



02, 



< K{p+ 1)(4T + ly/^ + (1 + p2) (^{62}) 



2 1n1/2 



(we have used fl5^ ). To derive (15^ from the resulting inequahty, it remains 
to note that 

(e{0^}) < (Ty'4rf ln(4T + 1) + 2. (43) 

Indeed, the coordinates of the Fourier transform of A[\..A^'*e are, up to fac- 
tor a, standard complex-valued Gaussian random variables, so that a~^Q'^ is 
the maximum of squared modulae of (4T + 1)^'^ of these variables; therefore 
E{Q^} < a2(4ciln(4T + 1) + 2) by Lemma [13 □ 
2*^. We now study the properties of the solution (p of problem ([6]). 

Lemma 17 One has 

|0|2T,2<2'^/V(2T + l)-'^/2. (44) 
|(l-0(A))e|;^,^<(l + 2V)eT; (45) 



E |(l-0(A))e|;^,, 



< (t(1 + 2 V) V 4^ + 1) + 2; (46) 



1(1 - 0(A)).|;^,^ < 1(1 - r(A)).|2T,2 + 2(1 + 2V)eT; (47) 

nl/2 



E (1(1 -0(A)). 



2 

2r.oo 



< 2(t(1 + 2V)v4t^ln(4T + 1) + 2 
+fi:(p+l)(4T+l)^/2_ 



(4J 



Proof. (HI); |0|2r,2 = |0|;t,2 < l0l2T,i < 2'^/V(2r + l)-'^/2, (the concluding 
inequality comes from the fact that cj) is feasible for (jH])). 
gS]), gi); We have 

1(1 - 0(A))e|^^,^ < (1 + |0|2T,i) max |Ar...A?e|;r,^ 

T;|r|<2i 



< (1 + (4r+ 1)'^/2|0|2T,2) max |A[\..A?e|^r, 

^ ' T:|r|<2i 

max I 

t:\t\<2T 



■■\r\ 

<(1 + 2V) max \Al\..A''/e\lT. 
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(we have used dH])). The resulting inequahty imphes that 



1/2 



E|(|(l-0(A))e|;^,. 
< (1 + 2V)f^\/4c/ln(4T + 1) + 2 



< (1 + 2V 



E{ max (|A[\..A^'*e|*2Too 

lr:|T|<2rV' 



1/2 



(we have used (H3|l ). 

(ESI); Note that the polynomial r defined in Lemma [16] is a feasible 

solution of the optimization problem (jS]) by the first relation in 0351) . so that 
the optimal value in the problem does not exceed J{r,y^j,). It follows that 



(a) J{$,ylT) < J{r,ylT) 

(b) |(l-0(A))?/|^^,^<|(l-r(A))|/|*^,^ 

(c) 1(1 - $iA))s\* < 1(1 - 0(A))e|*^,^ + 1(1 - r(A))y|; 



2T,oo 



id) 



+ 



E|(|(1-0(A)).|*^,, 
E{(|(l-r(A))y|*^,, 



1/2 



< 
1/2 



i^; (|(l-0(A))e|^^., 



nl/2 



Relation f H7|) follows from (c) combined with ( l38l) and ( l45l) (recall that p < p). 
Relation fHHj) follows from (d) combined with (146|) and fl39|) . □ 
3°. Our next step is to prove 



Lemma 18 One has 



((l-r(A))(l-0(A))s)J < |((l-r(A))s) 



+2'='/V(2T + ly^/^ 1(1 - r(A))s 



(49) 



2T,2 1 



(l-r(A))(l-0(A))s 



1/2 



< k{p+ 1)(2V + !)• 



(50) 



Proof. We have 



((1 - r(A))(l - 0(A))s) J < |((1 - r{A))s),\ + \ {${A){1 - r{A))s 

< |((l-r(A)).)J + |0|2T,2|(l-r(A))s|2^_2 

< |((1 - r(A)).)J + 2'^/V(2T + l)-'^/2 1(1 - r(A)).|2^_2 



see 
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as required in (H9|) . From the resulting inequality it follows that 







1/2 




1] 


< 









E{m-r{A))s)f} 

1/2 



E[ ((l-r(A))(l-0(A)). 
+2'/'p\2T + [e {1(1 - riA))s\l^^,} 

< Kip+ 1) + 2'^/V(2T+ l)-'^/2 [e{\{1 - riA))s\l^^,} 

< k{p+ 1) + 2'^/V(2T + l)-'^/2^(p + l)(4r + ly/^ 



1/2 



1/2 



[see (|41 
[see ([3] 



and dSO]) follows. □ 

4°. Now we are able to complete the proof of Theorem |H The error of the 
estimate s at the point t = is 



So - So = So - (0(A)|/)o = ((1 - 0(A))s)^ - (0(A)ej^ = + ^o , 



« = ((1-0(A)). 



e(2) 



0(A)e . 



(1) , .(2) 

' (51) 



Setting fr = e-r, we have 



.(2) I 



T:|r|<2r 



<2'^/V(2T+l)-^/Vlkoo, 
whence, by definition of 0^, 

lef^l <2'^/V(2r+i)-'^/^eT. 



< l0l2T,il/l2T,oo [see dZ 

[since is feasible for 



(52) 



Applying 0431) . we derive from the latter inequality that 



^{lef^l'}]'^' < 2^/Vp2(2T + l)-'^/y2dln(4T + 1) + 2. 



(53) 



We further have 



ro I 



< 



l-0(A))s 
(r(A)(l-0(A)) 



(l-r(A))(l-0(A))s 



a 



l2T,oo 



+ 



((l-r(A))(l-0(A))s)^ 



(54) 
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(the inequahty a is given by fl27|) . and h follows from the feasibility of (j) for 
(EI)), whence 



E 



{i-m))s 



2T,oo 



+ 



E 



(l-r(A)(l-0(A))s 



11/2 



<2'^/V(2T + l)-<^/2 
+fi:(p+l)(4T+ 1)^^/2 



2 V(l + 2 V) V ln(4T + 1) + 2 
+ k(p + 1)(2V + 1) 



1/2 



(55) 



(see dH]), dSHD). Combining ([SI]), ([SSD, (JSS]), we finally get 



[£; { I So - So I ' } ] < 2^^/ Vp2 ( 2T + 1) -'^/^ ^2dln(4T + l) + 2 
+2°'/V(2T + l)-"'/2 2V(1 + 2V)\/4rfln(4r+ 1) + 2 
+k{p+ 1)(4T + 1)H + k{p + 1)(2V + !)• 



(56) 



Recalling that p < p, n = e{2T + l)-^/^ ^nd that p > 1, dZD follows. 

Now assume that (s) is deterministic. In this case, from (l54l) combined with 

(H71) and (09]) imphes that 



|e[, 1 < 2i+'^/V(2T + l)-'^/2|(l - r(A))s|2T,2 
+2i+'^/V(l + 2V)(2T + ly^'^Qr + |((1 - ^(A))s)ol 



(57) 



while from (BTl), (USD it follows that 



1(1 - r(A))s|2T,2 < «:(p+ 1)(4T+ 1)^^/2 < 2'=^/2^(l + p), 
|((l-r(A))s)o| < k{1+p) < e(l + p)(2T+ l)-'^/2. 



(5J 



Therefore ( !57|) implies that 



ei'^l <33+V i^^ + per] (2T + 1) 



-d/2 



(59) 



Combining this relation with (!52i) and (I5T1) . we arrive at (IE]), n 
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5. 2. 1 Proof of Proposition 



In the proofs to follow, we focus on the case of well-filtered signals; the rea- 
soning in the case of well-predicted signals is completely similar. 

The case of m = 1 is evident. Now let m > 2, let T"*" be an integer, < < 
L+, and let T = \m~^T^\. Since G F^(^, p) and clearly T < L, there exist 
filters such that 

(a) : ord(g^) < T; (6) : \q^2 < Pj{2T + 

(c) : Ig^li = |g^|T,i < (2T + < pf, (60) 

(d) : [E{\si - iq^{A)s^\\^}f^ < 0,i2T + ly'/^ V(r : |r - t| < L). 

Now let filter q be defined by 

m 

Observe that 

ord(g) <mT < r+. (61) 



Note that 

\q\2 < 2>i...p™(2T + l)-^/2 < - lf/^2"'p,...pU2T+ + 1)-'^/^ (62) 



Indeed, we clearly have 



E(-l)'+' E g^M^)9'M^)-?''(^) 

«=1 l<jl<i2<---<><m 



< E E |g^^(z)g^^(z)...g^Hz)|2 < E E 

£=1 l<ji<j2<-<ji<m "-^^=1 l<ii<i2<..-<j^ 

a 

< [(i + Pi)...(i + p„)-i](2r + i)-'^/2 < 2-pi...p^(2r + i)-'^/2 



(2T+l)'*/2 



(a is by — c) since \u{z)v{z)\2 < |M|i|f I2, \u{z)v{z)\i < |M|i|t>|i], b is due 
to pj > 1), as required in fl62l) . Further, by fl60lc). for the filters 

Q'iz)=(mi-q'{z))] f ft (l-9'(^)) 
\i=i / \i=j+i 
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one has 



IQ^Il < (1 + Pl)...(l + P,-l)(l + + Prr.) < 



2"^-Vi...p„ 



(63) 



Now let r G Z'^ be such that \t — t\ < L+. We have 



E{\il-qiA))sl\'}f^" = 
< E |A,(l-g(A)y)J 

7=1 L ^ 



5:A,(l-g(A)y 

nl/2 



1/2 



a i=l 
m 

i=i 



1 - g(A))s^- 



nl/2 



g-'(A)(l-g^(A))s^' 



1/2 



< E[^{|A.f}]^/'IQ^|i max 



< 2""Vi-P™ 



r':|T'-T|<(m-l)T 



(l-g^(A)K , 



nl/2 



< 



(2T + ly^/"^ V 

Pj 



(2m-l)'^/22"^-Vi...p™E 



Pj 



(2T+ + 1) 



-d/2 



where a is due to independence of \j and {s^) and 6 follows from fl63l) . fiGOld). 
and since 

|r' - r| < (m - 1)T, |r - t| < L+ ^ |r' - t| < L+ + T+ < L. 

Combining the resulting inequality, (!6T!) . fl62l) and taking into account that 
T+ G {0, 1, ...jL"*"} is arbitrary, we conclude that s G F^+(6'"'", p"*"). Note that 
by construction, the filters certifying the latter inclusion are independent of 



5. 2. 2 Proof of Proposition 

(i): Let T < L, and let q be such that 



ord(g) < T, \q\2 < 



(2T+l)d/2 ' 



max 

T:\T-t\<L 



E{m-qiA))s)f} 



1/2 



< 



(64) 



(2T+lp72- 
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Let us set = exp{ia;'^r}g^, r G . Then ord(g) < T, |g|2 = \q\i and 

((1 - g(A))s)^ = exp{zKr + 

-E(exp{it^^r'}g^/)(exp{i[u;^(r - r') + 0]}s^_^,) 

r' 

= exp{z[c.^r + 0]}((l-g(A)).)^, 

so that flMl) remains vahd when g, (s) are replaced with g, (s). Thus, (s) G 
F|^(6',p). (i) is proved; (ii) is evident. □ 

5. 2. 3 Proof of Proposition [21 

Let T <L, and let q = (g^)^gz'' be such that ord(g) < T, \q\2 < p(2T + l)-'^/^ 

{|((1 - g(A))s) J'}] < ^(2T + e Z'^ : |r - t| < L). 

Setting .j.^^ = (2T + l)~('^^""'^g^j^ .j-^, we clearly have ord(g+) < T, |g+|2 < 
p(2T + l)-'^+/2 and 



E 



((l-g+(A))s^ 



1/2 



< ^(2T + V(r G Z^" : |r-t+| < L). 



It remains to note that 9{2T + l)-'^/^ < q+(^2T + for < T < L. □ 

5. ^. ^ Proof of Proposition O 

Let T < L, and let g' G Ct(Z^'), g" G Ct(Z^") be such that 

(a) : |g'|2 < p'(2T + l)-^'/2^ i^/'i^ < p"(2T + l)-'^"/2, 

4'(0 = E4'_.'g:.', k'-t'|<L, . (65) 

v' 

<"(0=E |r"-r|<L 

v" 

Let g(2;i, z^) = g'(zi, 2;d/)g"(2;d/+i, zi), so that 

g G CtIZ'^), |g|2 = |g'|2|g"|2 < pV'(2T + (66) 

(see (E3a)). Now let r = (r', r") be such that |r - (t',t")| < i^- We have 

(g(A)s(o).= E = e B'^,^^4.,s"Ai) 

{u' y')eZ''' xzd-" u'ez<i' 

= S;,(Os';»(0 = S(r',r"), 
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which combines with f l^ to yield that (s^) G '* \o,p'p"). □ 

5. 2. 5 Proof of Proposition [721 

We start with the following two evident facts: 

Lemma 19 Let (s^) G C(Z'^) be deterministic fields belonging to Y\{9,p), 
j = 1,2, ... such that s'l — > Sr, j — * oo, for every r G Z'^. Then (s) G F\{9,p). 

Indeed, for every T, < T < L, the fihers q^'^ G Cr(Z'^) which certify the 
inclusions (s^) G F\^{9, p) satisfy |q'-''"^|2 < p(2T + l)"'^/^ and therefore have a 
limiting point G Crjz'^) with |gJ|2 < p{2T + ly^/^. The filters {q'^}o<T<L 
clearly certify the inclusion (s) G F^(^^,p). □ 

Lemma 20 For every t G Z, the univariate exponential field {sr = exp{ujT}), 
uj eC, belongs to F^(0, 

Indeed, assuming 3?(cj) > and given T > 0, let us set q{z) = ^^[1 + 
exp{—uj}z~^ + exp{—2Lj}z^'^ + ... + exp{—Tuj}z'^]. Then q G Ct(Z), |g|2 = 
(T+l)-i/2 < 2V2(2r+l)-i/2, while clearly g(A)s = s. In the case oi^{uj) < 0, 
the same reasoning holds true for q{z) = + exp{ti;}2; + exp{2u}z'^ + ... + 

exp{Tuj}z'^]. □ 

To complete the proof, we need the following fact: 

Lemma 21 Let (s^) be a "simple" exponential polynomial - a deterministic 

exponential polynomial of the form (s^) = X) Qexpjct; (£)r}. Then 

i=\ 

WteZ'': (sr) G FUO,p,{N,,...,N,)), (67) 

where pd{-,...,-) is given by l[TS\) and Ni,...,N(i are the partial sizes of the 
polynomial. Besides this, the filters q^'^^ certifying the above inclusion can be 
chosen to depend solely on T and on the collection of the d sets Oj = {ujj{i) : 
£=1,...,M}. 

Lemma [2l] =^ Proposition [lOl - Assume first that the coefficients q in f|T5|) are 
deterministic. Since every one of the univariate functions f (t) = t^ , < k < m, 
is, uniformly on compact sets, the limit, as e ^ +0, of appropriate linear com- 
binations of the m + 1 exponents exp{— fee}, the exponential polynomial f[T^ 
is the pointwise, on Z'^, limit, as i ^ oo, of simple exponential polynomi- 
als (s^) with extended sets of "frequencies" {uj{£)}j^f. in the approximating 
polynomials, every one of these frequencies is replaced by {rrij + 1) frequen- 
cies ujj{i) — kei, < k < rrij. Note that by the definition of partial sizes 
of exponential polynomials, the approximating polynomials have exactly the 
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same partial sizes as the original polynomial (sr). Combining Lemmas [2T] and 
(fTOl) . we immediately conclude that the exponential polynomial (fT5|) belongs 
to F^(0, prf(A^i, A^rf)). Since the filters certifying well-filterability of 

the approximating polynomials (s^) can be chosen to depend solely on T 
and the sets of partial frequencies of these approximating poljTiomials, from 
the proof of Lemma [19] it follows that the filters g*^^-* certifying the inclusion 
(sr) G F^(0, pd{Ni, Nd)) can be chosen to depend solely on T and the sets 
of partial frequencies of [sr), as required in Proposition [TOl We have proved 
Proposition [TD] for the case of a deterministic exponential polynomial; since 
the filters certifying well-filterability of such a polynomial are independent of 
the coefficients q, the result is valid for random polynomials as well. □ 
Proof of Lemma Proof is by induction in d. 

Base d = 1 is readily given by Lemma [20] combined with Proposition [6] 

Step 1 < d ^ d + 1: Let Sr = J2ce exp{u^ {i)T} be a simple exponential poly- 

e 

nomial on Z'^'^^ with partial sizes Nj and the sets of partial frequencies Oj, 
j = 1, A^. Let T > 0, and let t G Z^~^^. By the inductive hypothesis, there 
exist filters g^^^ G Ct'(Z'^), h^'^^ G Cr(Z) (depending solely on T and on 
Oi,...,Od+i) such that 

(a) :|^?(^)|2<Pd(iVi,...,iV,)(2T+l)-'^/2, 
(a'):|/^(^)|2<Pi(iVd+i)(2r+l)-V2, 

(b) : = E r,-.gP Vr G V(r,) G E(Oi, O^), ^^^^ 
ih') ■.Pr=T. Pr-uh^P Vr G Z V(p,) G E(Orf+i), 



where E(0^ O"") is the space of all simple exponential polynomials on Z™ 
with the sets of partial frequencies 0\ O"^. Setting = g^^^^^^Ji^^^^^ , r G 
l/^^, we clearly have 

(69) 

= pd+i{Ni, ...,Nd+i) 



(see (]68la. a^). Further, for every (sr) G E(Oi, O^+i) we have, setting r = 
It', r") with r' G Z^, r" G Z: 
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(a is by ([6816') since {sr'^u',f,)fiez e E(Od+i), b is by (^b) since (s^,^//)^gzd G 
E(Oi, Od)), which combines with (!69|) to imply that 

(s,)GS*^(0,pd+i(iVi,...,iV,+i),r). 

Thus, the fihers g*^-^^ (which depend solely on T and Oi,...,0(i+i) certify the 
inclusion (s^) G L^(0, Pd+i(^i5 •■■5 ^d+i)- The inductive step is completed. □ 



5.2.6 Proof of statement in Remark [771 

It suffices to prove that for every nonnegative integer T and every m, d there 
exists a ffiter q'^'^\ ord(g*^^^) < T, depending solely on T,m,d, such that 



(a) q^'^\A)p = p for every polynomial (ITTll . 



This well-known fact can be proved by induction in d completely similar to 
the one used to prove Lemma [21] the only difference is in the Base, which now 
should be replaced with the following statement: 

m 

Lemma 22 Letp{T) = J2 Pi'^^ be a deterministic univariate algebraic polyno- 

mial of degree m. Then for every T > there exists a filter q G CtC^), depend- 
ing solely onT,m, with \q\2 < 16m(2T + such thatp{t) = ^q^^p{t — v) 

for all t & Z. 

Proof. By evident reasons, it suffices to prove that for a given T > there 
exists a collection of weights qt, —T < t <T, such that 

E = 1, E 9tf = 0,^ = l,...,m, E < - 

t=-T t=-T t=-T ZI -\- L 

By the standard separation arguments, this is the same as to prove that for 
every real algebraic polynomial r{t) of degree < m such that r(0) = 1 one 

T 

has J2 r'^{t) > ^^^2 ; or, which is the same, that for the real trigonometric 
polynomial p{(f)) = r(T sin(0)) one has 

9T -I- 1 

E/'(</>*)>^, </>* = asin(t/T). (71) 
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Note that the degree of the trigonometric polynomial p(-) is < m and that 
p(0) = 1. Besides this, p(0) = p{n — 0); due to the latter fact, 



M = max|p(0)| 



max |p(</))| > 



1. 



By Bernstein's Theorem on trigonometric polynomials, we have |p'(0)J^ < mM. 
Now let 0* G [— 7r/2,7r/2] be a point such that |p(0*)| = M, let A be the 
segment of the length — centered at 0*, and A be the part of this segment 
in [— 7r/2, 7r/2]. Note that the length of A is at least ^ and that for G A 
one has |p(0)| > |p(0*)| — ^i™^) ^ M/2. Let n be the minimum number 
of points (pt belonging to a segment 6 C [— 7r/2, 7r/2] of the length l/(2m), the 
minimum being taken over all positions of 6 in [— 7r/2, vr/2]. It is immediately 
seen that n > (1 — sin('7r/2 — l/(2m)))T — 2 > — 2, whence 



> 



M2 

> n > 

-4 - 4 



T 



- 2 



> 



1 



T 



- 2 



When T > GArn^, the latter quantity is > 



2T+1 
256m2 ' 



and in any case J2 P^i'Pt) 

t=-T 



T 

1. Thus, we always have P^i'Pt) > required in flTTj) . □ 



> 



5. ^. 7 Proof of Proposition [HI 

In the proof to follow, q stand for positive constants depending solely on D. 
1°. We start with the following evident observation: 

Lemma 23 There exists C\ such that for every polynomial p{t) of one variable 
satisfying the relation p{l) = 1 one has 

M < ciN, deg(p) < ciN, 

(s) G H*v(D) ^Sr = (p(D)s), V(r : |r - 1| < M). 
2°. Let us fix a positive integer A^, and let 

k 

6{uj) = J2 Wiexp{iuj^a{£)} : [-tt,tt]'^ 
e=i 

and let u be the normalized counting measure on i>{{u!}) = {2N+1)~'^, uj G 
Observe that in view of R.2 the function 5{-) maps into the unit disk 
= {C G C I Id < !}• Let p be the distribution of values of 5\^d , so that p is 
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the measure supported by the finite set M = | 3lj E Q% : ( = S^u)}, and 
/u({C}) = E z/({cj}). Let also =/i({C| 3?(C) > 1 -a}), tt > 0. 



Lemma 24 There exists C2 G (0, 1) such that 

M c M = {ci Id < 1, 1^^(01 < c,\i - mf'} 

F{a) < C2 '[a"'/' + N-'^], < a < 2. 



(74) 
(75) 



Proof. (El]), (175!) are evident when < 1, since then \S{u!)\ < 1 — C2 for 

properly chosen C2 and all u. Thus, in the sequel we focus on the case of 

k k 

E = 1 (recall that E < 1 by R.2). 
1=1 i=i 

2^.1) Let K = {u e [-77, vr]"' : 6{uj) = 1}. Since Pi > 0, EPe = ^ and 

e 

6{ij) = J2 P^exp{z0^ + Lo''^a(£)}, a point G K must satisfy the equations 



exp{i[(j)e + uj^a{i)]} = 1 V(l < £ < k), 



(76) 



whence 0^ + cj^a(£) G 27rZ V(l < £ < fc). Since Rank{a(£) : 1 < ^ < A;} = rf, 
the latter system of equations implies that K belongs to a set of the form 
r + ATj'^ with certain d x d nonsingular matrix A (depending solely on D). 
The cardinality of the intersection of latter set with the cube [— tt, tt]'^ does 
not exceed certain C3. Thus, CardK < C3. 
2°.2j Let G K, and let dcu G M" be such that \duj\<l. Then 



S{lj + du) = J2 P£exp{i[(f)i + u a{i)]} exp{i{duj) a{£)} 
1=1 

k 

= E peexp{i{dujya{i)} 



\6{lj + duj)\ 



< 

b 



J2 Piexp{i{duj)'^a{£)} 



E pJ 1 + i{dufa{i) - i ({dujfa{i) 
'.=1 \ ^ 



+ c^ldiul 



+ C4^\duj\^ < 1 — Csldcup + C4|(ico'| 



(for a, see (176|) . 6 is by (l20l&). c is due to Rank {{a{i)}e) = d). It follows that 
with properly chosen Cg one has 



V(u; G [-TT, tt]^, |(5(u;) - 1| < a) G K : |u; - cu] < Cg ^ v^. 



(77) 
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Since Card (K) < C3 by 2°.l) and |(5(co')| < 1 for all cj, we conclude that 

V ({cu G ^% : |(5(cu) - 1| < a}) < Cyfa^/^ + iV"'^] Va < 2. (7J 



2°. 3; Now we can complete the proof of (jZl]), Let G K, rftu G M^, 

|(ia;| < 1. We have 

_ k 

+ (iu;) = p^exp{i[0£ + co' Q;(£)]}exp{z((iti;) «(£)} 

= E p^exp{i(rfcj)^a(£)} = E p/l + 
^^£=1 £=1 V 

a 

~\ ({dujfa{^)'' - i + r,(a;,d^)), (79) 

[|r^(u;,(iu;)| < Ciolrfcc^l^] 
^gP£ (1 - \ ((rf^)^a(^))' - I ((cio;)^a(£))' + r,{uj,duj) 



(for a, see fl76l) . for 6, see fl20l) ). Taking into account that E P£ = 1 and 

Ciilrfcup < E P£ (('^^)"^«(^) ) ^ Ci2|(icijp, we conclude from flTTI) combined 
with fl7^ that for properly chosen C13 one has 

^ G [-vr, tt]^ ^ ^(5(0^))! < ci3(l - ^{K^))f'\ 

and dZH) follows. By ^ one has |1 - b{yj)\ < 014(1 - 3?(5(c^))), so that (1751) 
follows from ([78]). □ 

3°. Let n be a positive integer, and let T„((^) be the Tschebyshev polynomial 
of degree n. Recall that this polynomial is defined as follows: 

vf^ + w~"' 

Tn{C) = , where w = C + i^T^C^. (80) 



In ( IHOj) . the choice of the branch of ^/'■ affects the value of w, but does not 
affect the value of + since we intend to work with ( from the unit 
disk, so that 3fJ(l — C^) > 0, in the calculations to follow we deal with the main 
branch of in the closed right half-plane. On the segment [—1, 1] of the real 
axis one has Tn{() = cos(n acos(C)), whence T„(l) = 1, T^(l) = n^. From 
these relations it follows that the function -Pn(C) = l^i'^i^f) is a polynomial of 
degree n — 1, and P„(l) = 1. 
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Lemma 25 One has 



Pn{a) = max{|P„(C)| : C e M, 3^(0 = 1 - «} 

'ci5, 0<a<^ (81) 

C15(1+C15«)" 1 <^<2- 



< gn(a) 



Proof. Let C = 1 - a + e M, so that 

< ci6«'/'. (82) 

We have 



w = C + i^T^ =l-a + il3 + i^2a - - 2i(l - a)^ + /?2 

= 1 - a + + zv^Y^l - 0.5a + [0.5/3 - z(l - (83) 

= l + iv^ + ri(C), |ri(C)|<ci7a 



(since |/3/a;| < ci6a/« by (!83|) ). Note that completely similar considerations 
demonstrate that 



w 



1 = C - - = 1 - + ^2(0, k2(C)l < Cl7«- 



B4) 



S'^.lj Assume, first, that < a < In this case from (!83l) it follows that 
|1 — ^1 < v^'^"^, whence, taking into account (j83l) . 

I^n - (1 + n(«; - 1) + ^^(w; - 1)2)| < Ci7(n|ti; - \\f < Cisn^a^/\ 
\yj-n _ (1 _ n{w - 1) + ^^(u; - 1)2)1 < c^,{n\w - 1|)3 < ci8n3a3/2 



w +w 



< — 1|2 + cisn^a^/^ < Ci9(?22a + n^a^/^) < 020^20. 



Thus, one has |Pn(C)| 
< a < 4. 



n^\a—ip\ 



< C15, as required in (IHTi) for the case of 

3°.2j Now consider the case of ^ < a < 2. From (IHSl), (IHl it follows that 

< 



|w| < 1 + C2ia, \w """I < 1 + C2ia, whence |Pn(C)l 
as required in (|8T]) . □ 

4°. Let Q{C) = It is immediately seen that 



rfi\oi—il3\ 



C22(1+C21Q)" 



C = l-a + z/5GM^|Q(C)|<l-C23a [C23 < 3] 



^5) 
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Now let C24 be a positive integer which is > — (see ( IHTj) ). Consider the 
polynomial ^„(C) = P„(C)Q^^^"(C)- 

Lemma 26 For every positive integer n, the polynomial Sn{C) possesses the 
following properties: 



(a) : deg(5'„) < 025^; (b) : S'„(l) = 1; 

(86) 

(c):max{|5„(C)|:CeM, 3?(C) = l-a}<Ci5min ' ^ 



Proof. Relations (!86l a — b) are evident (take into account that -Pn(l) = 1 and 
deg(P„,) < n). To verify (l86l c). note that if( = l — a + i^E M, then in view 
of flHTl) one has 



0<a<^^ |5„(C)| < |Pn(C)IIQ(C)l'=^^" < \Pn{0\ < C15; 

^<a<2^ |5„(C)| < |P„(C)IIQ(C)r^^"^ci5^i^fif^(l - C23«)'^-" 

a 

^ exp{ci5na} r C15 

S Ci5 ^^2^j exp|— C23C24"-a| S ^ 



(for a, see ( 185|) . 6 is due to C23C24 > C15). □ 

5°. Now we are ready to complete the proof of Proposition [121 Given a positive 
integer n, let us set -R„(C) = S^iO- view of (186|) one has 

(a) : deg(i?„) < C2en; (b) : i?„(l) = 1; 

(c) : max{|i?„(C)| : C e M, 3?(C) = !-«}< r„(a) (87) 



= C26 mm 
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Consider the filters g(")(2) given by g(")(A) = K(D), n = 0, 1, ... By (EHfe) 
and Lemma [251 we have 



ord(g('^(^))) < T, 

= (gW^))(A)s), V(r : |r - t| < C27N). 



T < C27N 
1 < n{T) = [C27TJ 



fs) G H* (D) 



By Parseval's equality, we have also (in what follows, n = n(T)) 



= j \RM^))\Mduj) = j |i?„(C)P^(t/C)^ j ^dF{a) (89) 
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with a given by (IHTlc). (17il) and the definition of F{-). Let 7 be the measure 
on [0, 2] defined by G{a) = 7([0, a]) = c^^a'^^^ + iV"'^), so that 

< = 7([0,a]) Vae[0,2] (90) 



(see (US])). We have 



Pnia)dF{a) = pn{2) - p'^{a)F{a)da < p„(2) 



p^{a)G{a)da = p„(2) - p„(2)G(2) + / pn{a)-f{da 







^ y p„(a)7(da) ^ 

6 c 
+Pn(0)iV-'^ 



n-^'^a-^ 1 



C28 y mm 



^C3o [N-'^ + n-''] <C3i(2T+l) 



a 2 ^c^a 



(91) 



(a holds since Pri(-) is nonincreasing, see flHTlc). and by fl90l) . 6 holds since C2 G 
(0, 1), see Lemma [211 c is by flHTIc) and fl89l) . ci is due to n = n(T) = [C27TJ). 
Combining fl89|) and fl9T]) . we conclude that 

|g^"^^^^|2<C32(2T+l)-'^/2_ (92) 



From ( |88ll and (l92l) we conclude that if L = [cssA^J and T < L is such that 
n{T) = [C27TJ > 1, then 

. , |g(^)|2<C32(2T + l)-^/2, 

Sr = (g(^)(A)s), V(r, |r - 1| < L, (s) e H*v(D)) 



(indeed, one can choose, as a required q^^\ the filter g*^"*^-^))). Setting q^'^\z) = 
1 for T < — , we enforce the validity of (p3|) for all T, < T < L. Thus, 
H*v(D)cfX,,^j(0,C34). □ 



5. 2. 8 Proof of Proposition [73l 

Lemma 27 Lei / G H'*"(M) he a deterministic function, let N < M/2, and 
let t G I/, \t\ < N. Consider the "discrete box" 5^ = {r G Z<^ : |r - t| < A^}, 
and let cj) he a deterministic function on -B^ which coincides with f on the 
"discrete boundary" dBj^ = {r E Z'^ : \t — t\ = N} of Bj^ and is "discrete 
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harmonic" : t G Z*^, |r — t| < ^ 0^ = ^ J2 (pr+e- Then 



=(n.---.fd) 
lnl=---=l'!dl=i 



TeB%^ |/(r) - 0.1 < ci||/|U,2MiV-' (94) 



(from now on, Ci are positive absolute constants). 

Proof. First, we should prove that the "discrete harmonic" function on 
which coincides with / on dB\^ does exist. This fact is well known; we 
present here its proof just for the sake of completeness. Let ■0 be a function 
on dB^j^. Consider the following random walk on 5^: arriving for the first 
time at a point r from 95^, we pay penalty 0'(r) and terminate; from an 
"interior point" r G int-B^ = B\q\dB\j we make a random step of length 1 
along one of the coordinate axes, choosing every one of 2d possible steps with 
probability l/{2d). It is immediately seen that the expected penalty payed at 
the termination, treated as a function of the initial state, is a discrete harmonic 
function with the boundary values ifj. 

Now, since |t| < and 2N < M, the function / is harmonic in the "continuous 
box" Di,j^ = {r G M°' : |r-t| < 2N}, and the uniform norm of / in this square 
does not exceed ||/||oo,2A/- From the standard results on harmonic functions it 
follows that 

< C2||/||oo,2MiV-^ = 1, 2, 3, 4, J = 1, d. (95) 



Consequently, for the basic orths e^, j = 1, c? we have 

<C3|s|i/|U,2A/iV-'. 

Since / is harmonic, we conclude that 

|(D/).| < C4||/||oo,2MiV-', r G B%. (96) 



V(r G D%) : 



dx 



r G 



Af5 



\s\ < 1 



fir + se,) - Y: 



1 d"" 



Now let h = f\^, - G C(fi^) and let = ± ^^^Ma.,2M ^ _ ^^y^ 

Taking into account fl96l) and the fact that is discrete harmonic, we have for 
T G int5^: 

(D/.+). = (Bhl + MnpIE > 0, (Bh-)^ = (Bh)^ - Mlh^ < 0, 

whence both the maximum of and the minimum of h~ over are attained 
at dBjf. Since at the discrete boundary of Bj^ we have / = and therefore 
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/i+ < 4c4||/||oo,2mA^ ^, we conclude that r G B% =^ K < < max < 



2o?C4||/||oo 2A'/^ ^- By similar reasons, r G -B^ =^ h^- > > min /i^ > 

-2dC4||/||oo,2MiV-'. □ 

Now let It] < M/8 and L < M/8. Given T, < T < L, and applying 
Proposition [121 we can build filter g(^) G ^^(Z'^) such that 

|g^^^|2 < c,{2T + 0. = E V(r : |r - t| < L) (97) 



for every (p which is discrete harmonic in the discrete box B^i- Now let 
/ G H(M, i?). Applying Lemma [271 we can find function which is dis- 
crete harmonic in the box i?2L such that \(f)r — /rP < Cg||/||^ 2Af-^~^ 
r G -B2L- From ( jOTj) it now follows that 



V(r : |r-t| < L) : 



. I W\<T 



1/2 



< C6 [i5{||/llL,2M}]'/' L~\l + < C6i?L-2(l + |g(^)b(2r + 1) 



<R 



(recall that d<A). □ 

The proof of Proposition [I3| is completely similar to that of Proposition [TOl 
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